Determining the quality of mathematical software using reference data sets

This paper describes a methodology for evaluating the numerical accuracy of software that performs mathematical calculations. The authors explain how this methodology extends the concept of metrological traceability, which is fundamental to measurement, to include software quality. Overviews of two European Union-funded projects are also presented. The first project developed an infrastructure to allow software to be verified by testing, via the internet, using reference data sets. The primary focus of the project was software used within systems that make physical measurements. The second project, currently underway, explores using this infrastructure to verify mathematical software used within general scientific and engineering disciplines. Publications on using reference data sets for the verification of mathematical software are usually intended for a readership specialising in measurement science or mathematics. This paper is aimed at a more general readership, in particular software quality specialists and computer scientists. Further engagement with experts in these disciplines will be helpful to the continued development of this application of software quality